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Cost-Benefit  Analysis  of 
Computer  Graphics  Systems 

Ira  W.  Cotton 


This  report  assesses  the  state-of-the-art  in 
cost-benefit  analyses  of  computer  graphics  systems  and 
suggests  an  approach  for  developing  improved  methodology. 
Cost-benefit  analyses  are  distinguished  from  analyses  of 
system  performance  in  that  the  latter  are  directed  at 
optimizing  system  performance  at  a  given  level  of 
investment,  while  the  former  are  directed  at  justifying 
the  investment  itself. 

Computer  graphic  system  design  alternatives  are  first 
outlined.   Then  methods  of  analyzing  the  performance  and 
costs  of  computer  systems  in  general  and  graphic  systems 
in  particular  are  discussed.   With  this  information  it  is 
shown  how  cost-effectiveness  analyses  may  be  performed. 
The  next  crucial  step  is  to  conduct  benefit  analysis,  an 
ill-defined  art.   The  results  of  benefit  analysis  must  be 
combined  with  cost-effectiveness  analysis  in  order  to 
perform  the  desired  cost-benefit  analysis. 

An  experimental  methodology  is  suggested  for  better 
performing  benefit  analyses  of  computer  graphics  systems. 
A  more  rigorous  formulation  of  the  cost-benefit  procedure 
is  then  outlined.   No  attempt  is  made  in  this  report  to 
actually  perform  such  an  analysis. 


Key  words:  Computer  graphics,  cost-benefit  analysis, 
cost-effectiveness,  economics,  performance 
evaluation 


1.  Introduction 


Computer  graphics  is  a  sub-discipline  within  computer 
science  which  deals  with  the  manipulation  of  digital 
representations  of  pictorial  information,  including  output 
of  the  picture  on  an  appropriate  display  device  and 
modification  of  the  image  in  response  to  operator-initiated 


actions.   There  are  many  systems  based  on  this  technology  in 
use  today,  primarily  in  the  area  of  computer-aided  design, 
and  many  cost  savings  are  attributed  to  such  systems.   Hard 
evidence  to  support  such  claims  is  mostly  lacking,  however. 

Few  studies  have  been  published  relating  economic 
principles  to  the  design  and  implementation  of  computer 
graphics  systems.   Those  that  have  been  published  deal  with 
only  one  side  of  what  is  essentially  a  two-faceted  problem. 
Either  a  detailed  analysis  of  the  costs  and  performance  of 
an  optimized  system  is  presented,  without  corresponding 
analysis  of  the  economic  benefits  to  be  derived;  or   expected 
benefits  are  estimated  without  discussion  of  the  costs  of  a 
system  to  achieve  those  savings.   The  first  type  of  study 
may  be  characterized  as  a  cost-effectiveness  evaluation;  the 
second  as  a  benefit  analysis. 

Cost-effectiveness  studies  are  aimed  at  optimizing  the 
performance  of  a  system  according  to  stated  criteria  and  for 
a  given  level  of  investment.   The  sensitivity  of  system 
performance  to  changes  in  level  of  investment  may  also  be 
considered  within  the  scope  of  cost-effectiveness  analysis. 
No  value,  however,  is  assigned  to  the  benefits  from  improved 
performance.   Benefit  analysis,  on  the  other  hand, 
investigates  just  this  issue,  the  value  of  improved 
performance,  but  without  regard  to  the  cost  of  achieving  it. 

Cost-benefit  analysis  provides  a  link  between 
cost-effectiveness  analysis  and  benefit  analysis. 
Cost-benefit  analysis  is  concerned  with  providing  adequate 
information  for  making  the  investment  decision.   Is  the 
stream  of  benefits  resulting  from  the  use  of  a  system  with  a 
given  level  of  performance  greater  than  the  required 
investment?   What  is  the  optimal  level  of  investment  to 
maximize  net  benefit?   Such  are  the  questions  addressed  by 
cost-benefit  analysis. 

Cost-effectiveness  analysis  is  necessary  but  not 
sufficient  for  cost-benefit  analysis.   A  valid  cost-benefit 
study  requires  that  the  optimal  level  of  performance  for  a 
given  level  of  investment  be  considered.   Alternatively 
stated,  for  each  given  level  of  investment,  the  value  of  the 
benefits  considered  should  be  the  maximum  possible.   This  is 
precisely  the  information  provided  by  cost-effectiveness 
studies.   What  cost-effectiveness  studies  fail  to  provide  is 
any  assessment  of  the  value  of  the  benefits.   Such 
assessments  are  often  difficult  to  perform,  but  they  are 
crucial  to  the  investment  decision. 

We  will  be  concerned  in  this  report  with  the  methodology 


of  cost-benefit  analysis.   Methodologies  for 
cost-effectiveness  studies  will  be  considered  too  because, 
as  has  just  been  explained,  they  are  essential  for  valid 
cost-benefit  studies.   The  relevant  literature  will  be 
critically  reviewed  and  suggestions  offered  where  published 
accounts  are  lacking.   No  attempt  is  made  here  to  actually 
perform  such  an  analysis. 


2.  Computer  Graphics  System  Design  Alternatives 


Before  any  consideration  of  economic  analysis  is  possible 
it  is  first  necessary  to  outline  the  technology  of  the 
systems  under  consideration:   It  is  convenient  to  assign 
graphic  systems  to  one  of  three  main  classes,  as  follows. 


Basic  System 

Computer  graphics  systems  of  the  type  under  analysis 
consist,  in  their  simplest  form,  of  an  analog  display  device 
similar  to  an  oscilloscope,  a  display  processing  unit  which 
converts  digital  display  commands  to  analog  signals  to  drive 
the  display,  and  a  general  purpose  computer  system  which 
generated  the  digital  commands  in  accordance  with  some 
application  program.   This  system  is  illustrated  in  Figure 
1. 


Satellite  System 

This  basic  system  may  grow  in  a  number  of  ways  [22], 
More  and  more  features  may  be  added  to  the  display  processor 
until  it  comes  to  resemble  a  computer  itself*   This  permits 
the  display  device  to  be  located  remotely  from  the  main 
computer,  connected  by  a  communications  link.   Since  the 
link  is  generally  slower  than  the  processing  rate  of  the 
display,  local  storage  is  required  to  hold  the  commands  sent 
from  the  central  computer.   However,  in  this  type  of  a 
system  we  permit  only  primary  storage;  secondary  or  mass 
storage  such  as  a  disk  or  drum  is  not  permitted.   This  type 
of  configuration  is  called  a  satellite  graphics  system 
because  the  graphics  processor  is  located  at  a  distance  from 
the  central  computer  but  is  dependent  on  it.   The  general 
configuration  is  illustrated  in  Figure  2. 

Such  a  system  offers  a  number  of  tradeoffs  for  the 
designer  [25].   The  speed  of  the  communications  link,  the 
power  of  the  display  processor  and  the  amount  of  local 
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Figure  1.  Basic  Computer  Graphics  System 
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Figure  3.  Network  Graphics  Configuration 


primary  storage  all  may  be  increased  at  increased  cost, 
Also,  certain  tasks  may  be  performed  either  in  the  main 
computer  or  in  the  local  processor.   The  decision  as  to 
which  components  to  upgrade  and  where  to  perform  processing 
tasks  is  not  obvious. 


Network  Configuration 

The  situation  can  get  even  more  complex.   It  is  well 
within  the  capability  of  today's  minicomputers  to  control 
multiple  displays  located  remotely  from  the  central 
computer.   In  such  configurations,  it  is  again  desirable  to 
separate  the  digital  processing  functions  from  the  analog 
conversion.   In  addition,  it  will  probably  be  necessary  at 
this  stage  to  permit  the  graphics  processor  to  have  a  mass 
storage  subsystem.   All  of  this  results  in  a  true  network 
coflf iguration  where  multiple  users  share  facilities  at  each 
end  of  the  communications  link.   This  is  illustrated  in 
Figure  3- 

The  attachment  of  input  devices  has  not  been  considered 
in  order  to  simplify  the  problem.   In  general,  input  devices 
send  their  data  to  the  closest  processor.   In  the  first  case 
this  would  be  the  main  computer.   In  the  latter  two  cases 
input  signals  would  be  initially  processed  by  the  satellite 
computer.   The  degree  of  processing  performed  at  the 
satellite  in  response  to  inputs,  however,  is  a  design 
variable  and  may  vary  widely  from  system  to  system  [7], 


Tradeoffs 

For  each  of  these  types  of  systems  there  are  three  types 
of  tradeoffs  which  can  be  made:  hardware/hardware, 
hardware/software  and  software/software.   Hardware/hardware 
tradeoffs  refer  to  the  possibility  of  diverting  costs  from 
one  component  to  upgrade  another.   For  example,  mass  storage 
capacity  at  a  satellite  might  be  reduced  in  order  to  pay  for 
a  faster  data  link,  or  vice  versa.   Hardware/software 
tradeoffs  refer  to  the  possibility  of  implementing  certain 
functions  either  in  hardware  or  software.   Character 
generation  is  a  good  example  of  a  function  which  can  be 
provided  either  way.   Software/software  tradeoffs  refer  to 
the  possibility  of  performing  certain  processing  tasks 
either  in  the  main  computer  or  the  satellite.   Display 
generation,  or  the  process  of  formatting  display  commands 
from  a  description  of  the  graphic  design  in  a  data 
structure,  is  an  example  of  a  function  which  may  be 
performed  in  either  place.   All  of  these  types  of  tradeoffs 


challenge  the  system  designer  to  come  up  with  the  optimum 
for  a  particular  set  of  criteria.   This  is  what  the  art  of 
system  design  is  all  about,  and  efforts  to  express  these 
tradeoffs  analytically  is  what  is  meant  by  making  system 
design  into  a  science. 


3.  Analysis  of  System  Performance 

There  are  essentially  two  basic  approaches  to  evaluating 
the  performance  of  a  computer  system:  observation  and 
analysis  [30].   The  system  is  either  set  various  tasks 
(benchmarks)  to  perform  and  is  graded  on  how  well  it 
performs  them;  or  the  system  is  analyzed  in  terms  of 
specific  parameters  in  order  to  derive  a  measure  of 
performance.   The  tradeoff  between  these  two  approaches  is 
between  the  detail  and  accuracy  of  the  analysis  and  the  cost 
of  performing  it.   The  first  method  has  the  appeal  of  direct 
measurement:  the  system  is  assessed  as  a  whole.   However, 
benchmarks  are  expensive  to  prepare  and  to  run.   Parametric 
analyses  are  less  expensive  to  perform,  but  their  results 
are  less  reliable  since  the  effects  of  small  errors  in 
determining  the  parameters  may  be  cumulative.   Specific 
techniques  of  system  analysis  fall  somewhere  on  a  continuum 
between  pure  observation  and  pure  parametric  evaluation. 


Feature  Analysis 

The  first  serious  attempt  to  provide  a  formal  basis  for 
comparing  computers  was  a  listing  of  the  various 
characteristics  of  different  computers  published  by  Adams 
Associates.   This  listing  has  been  called  an  Adams  chart;  it 
is  still  published  by  Keydata  and  typifies  a  simple 
portrayal  of  the  classical  features  of  computers 
manufactured  worldwide.   In  the  graphics  area,  Adams 
Associates  also  published  the  <Computer  Display  Review> 
which  listed  the  features  of  graphics  equipment;  it  was 
continued  by  Keydata  and  is  now  published  by  GML 
Corporation. 

As  pointed  out  in  [17],  the  limitation  of  the  Adams  chart 
is  the  inference  that  performance  can  be  meaningfully 
predicted  from  these  classic  features.  It  has  been  well 
established  that  the  use  of  such  raw  speed  parameters  as 
clock  speed,  arithmetic  speed,  memory  speed,  word  size  or 
I/O  rate  can  be  misleading  in  predicting  comparative 
performance  between  different  systems.   Over  the  years,  many 
people  have  taken  different  combinations  of  these  classic 


features  and  have  used  these  combinations  as  figures  of 
merit  to  infer  performance  measures. 

Any  listing  of  simple  features  fails  to  incorporate  the 
differences  that  each  system  designer  included  to  improve 
his  system.   The  power  of  each  instruction  set  combined  with 
each  system's  architecture  is  not  indicated;  the 
effectiveness  of  system  software  is  not  considered;  and 
factors  for  evaluating  special  capabilities  such  as 
multiprogramming  or  multiprocessing  are  lacking. 

Instruction  Mixes 

A  major  advance  over  the  feature  analysis  approaches  was 
the  development  by  the  Auerbach  Corporation  of  a  standard 
set  of  small  problems  which  could  be  programmed  by  experts 
for  the  variety  of  different  machines  and  configurations 
under  consideration.   The  results  yielded  the  amount  of 
memory  required,  the  amount  of  code  and  similar  detail.   The 
implication  is  that  performance  on  these  standard  problems 
is  indicative  of  the  performance  on  actual  user  problems, 
and  that  small-problem  performance  is  indicative  of 
large-problem  performance. 

As  Johnson  observes,  however,  all  instruction  mixes  [17] 
suffer  from  the  limitations  that  the  actual  mix  in  use  may 
or  may  not  correspond  to  that  of  the  "standard"  mix. 
Further,  none  of  the  instruction  mixes  yet  devised  provide  a 
reliable  indication  of  full  load  behavior  of  computing 
systems.   This  is  an  especially  serious  shortcoming  when 
attempting  to  predict  the  performance  of  multiprogramming  or 
multiprocessing  systems.   The  full  load  performance  of  such 
systems  does  not  appear  to  be  proportional  to  its 
small-problem  performance.   Nevertheless,  simple  instruction 
mixes  have  been  widely  used  over  the  years  because  they  are 
simple,  easy  to  use,  and  provide  a  reproducible  evaluation 
methodology. 

The  application  of  this  technique  to  the  analysis  of 
graphic  systems  is  reflected  in  the  set  of  test  patterns 
which  was  developed  by  Adams  Associated  for  their  <Computer 
Display  Review>.   These  patterns  were  designed  to  be  coded 
for  each  display  system  to  be  evaluated.   One  immediate 
result  would  be  a  comparison  of  the  number  of  words 
necessary  to  code  the  pattern  in  its  entirety.   The  results 
of  running  each  pattern  were  the  percentage  of  the  pattern 
that  could  be  displayed  flicker  free  -  a  measure  of  the 
capacity  of  the  display  for  applications  of  which  the  test 


patterns  were  supposedly  representative.   The  patterns  used 
were  the  following: 

Alphanumeric  Test  Pattern  -  The  alphanumeric  test 
pattern  is  representative  of  applications  requiring  the 
display  of  tabular  or  free-form  character  data-   Figure  4 
illustrates  this  test  pattern- 
Weather  Map  Test  Pattern  -  The  weather  map  test 
pattern  is  representative  of  a  class  of  applications 
including  geographical  or  mathematical  contour  mapping  as 
well  as  sheet  metal  styling.   Figure  5  illustrates  this 
test  pattern. 

Graph  Test  Pattern  -  Graphs  and  charts  are  very  common 
displays,  both  in  business  and  scientific  applications. 
Figure  6  illustrates  this  test  pattern. 

Architectural  Drawing  Test  Pattern  -  The  architectural 
drawing  test  pattern  is  representative  of  applications 
where  a  large  quantity  of  unstructured  data  must  be 
displayed  with  high  precision.   Such  applications  include 
circuit  board  layout  and  mechanical  design.   Figure  7 
illustrates  this  test  pattern. 

Electronic  Schematic  Test  Pattern  -  The  electronic 
schematic  test  pattern  is  representative  of  applications 
where  pictures  are  constructed  hierarchically,  using 
repeated  instances  of  certain  patterns  or  subpictures. 
Such  applications  include  logical  design  and  certain 
layout  problems  and  mechanical  analysis  systems.   Figure 
8  illustrates  this  test  pattern. 


Kernels 

A  more  sophisticated  version  of  the  instruction  mix 
approach  to  the  anlysis  of  system  performance  uses 
"kernels."   A  kernel  is  a  complete  nucleus  problem; 
meaningful  kernel  problems  are  selected  according  to  the 
type  of  application  of  interest,  e.g.,  a  matrix  inversion 
for  scientific  problems  or  a  small  sort  for  business 
applications.   The  execution  time  of  a  set  of  such  problems 
is  assumed  indicative  of  the  system's  execution  of  the  whole 
application.   When  kernels  are  programmed  by  experts  on 
their  respective  machines  and  the  actual  software  of  that 
system  is  used,  the  results  can  be  good  indicators  of  that 
small-problem  behavior  of  that  system. 
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System  Valves       System  Status       System  Signals 

SUPERHEATER       C      *TRIPPED*      STEAM  PRESSURE     N 
REHEATCR  C  FURNACE  PRESSURE   N 

MAIN  GAS  SIIUTOFF   0  GAS  TEMPERATURE    N 

VENT     0  COMBUSTION  ■        M 

MAIN  OIL  SirUTOFF  C  FORCED  DRAFT  FAN  R 

RETURN   C  AIR  HEATER  0 

NORTH    C 

SOUTH    C  IGNITOR  CHARGED  Y 

GAS  READY  Y 
IGNITOR  SinJTOFF    0 

VENT      C  OIL  READY  N 

DUAL  PERMISSIVE  N 
PENTHOUSE  NO.  1   0 

NO.  2   0  MODE  SWITCH  C 

VENT    0  PURGE  REQUIRED  Y 

BURNERS 

1234S6789  10 

STATUS  S  *  *  *  I  - 

IGNITOR 

FLAME  ----__-_-_ 

TRANS  ----______ 

SHUTOFF  CCCCCCCCCC 

MAIN  FIAME  N  F  F  F  -  -  -  - 

GAS   SHUTOFF     CCOOOOCCCC 

OIL  SHUTOFF  CCCCCCCCCC 

UNIT  OOOOOOOOOO 

ATOM  IIIIIIIIII 

PURGE  CCCCCCCCCC 

COOL  CCCCCCCCCC 

DAMPER  OOOOOOOOOO 

CONTROL 

IGNITOR      ---------- 

GAS-  -XXXXX 

OIL      ---------- 

DUAL      -  -  -  -  -  -  -  -  -    "      - 

OFF     XX  -  -  »-  -  -XXX 


Figure     4 

ALPHANUMERIC  DISPLAY  TEST  PATTERN 
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Figure    5 
WEATHER  MAP  TEST  PATTERN 
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Figure  7 


ARCHITECTURAL  DRAWING  TEST  PATTERN 
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Figure    8 
ELECTRONIC  SCHEMATIC  TEST  PATTERN 
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Benchmarks 
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actual  application.   Such 
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The  real  limitations  of  the  benchma 
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to  prepare  realistic  data.   The  task  i 
batch-oriented  computer  systems  for  wh 
programs  written  in  standard  compiler 
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for  graphics  systems  which  are  truly  r 
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Simulation 


A  final  technique  which  can  be  used  used  in  performance 
analysis  is  simulation.   Simulation  cuts  across  the  two 
general  approaches  of  observation  and  analysis.   To  use  this 
technique,  a  model  is  constructed  of  the  system  under 
consideration  and  the  performance  of  this  model  is 
"observed"  under  the  desired  conditions.   The  model 


*  Work  is  in  progress  by  a  number  of  researchers  at  The 
Mitre  Corporation  and  at  the  National  Bureau  of  Standards  on 
"Terminal  Environment  Simulators"  which  can  serve  as  drivers 
for  benchmark  programs  for  interactive  systems. 

14 


represents  a  simplification  of  the  actual  system  which  is 
constructed  from  analysis  of  the  system.   If  the  model 
accurately  represents  all  the  salient  features  of  the 
actual,  when  it  is  run  with  the  desired  (simulated)  workload 
the  result  is  analogous  to  observing  the  actual  system. 

Of  course,  the  results  of  simulation  can  be  no  better 
than  the  many  assumptions  that  go  into  the  construction  of 
the  model  and  into  the  design  of  the  simulator. 
Constructing  a  computer  system  simulator  for  performance 
evaluation  purposes  is,  as  has  been  pointed  out,[H]  not  an 
easy  task.   If  the  level  of  simulation  is  too  fine,  the 
simulator  will  be  too  costly  to  use.   If  the  level  is  too 
gross,  not  enough  information  will  be  yielded.   Despite 
these  difficulties,  simulation  is  becoming;  much  more  widely 
used,  and  a  good  simulator  properly  used  nay  be  one  of  the 
better  tools  for  predicting  system  performance  accurately. 
As  we  shall  see  presently,  it  is  also  a  technique  which  can 
and  has  been  used  to  analyze  computer  graphics  systems. 

*J .  Cost  Analysis 


The  principal  problem  of  the  cost  analysis  of  computer 
systems  is  to  express  all  the  costs  of  alternative  system 
proposals  in  a  form  suitable  for  analysis.   This  requires 
that  ALL  relevant  costs  be  included,  and  that  all  costs  be 
expressed  in  a  comparable  form.   The  question  of  which  costs 
to  include  has  been  well  treated  in  the  literature.   Joslin 
[18]  presents  a  detailed  analysis  of  all  the  types  of  costs 
which  must  be  included  in  evaluating  a  system  design 
alternative.   The  main  complication  in  many  cases  is  the 
need  to  include  development  costs  in  the  analysis  which  are 
not  known  in  advance  but  which  must  be  estimated. 


Cost  Estimates 

If  two  or  more  proposed  systems  are  to  be  compared,  it  is 
necessary  to  compare  the  amount  of  development   effort 
required  for  each.   Data  for  such  an  analysis  is  "softer" 
than  data  describing  the  pricing  of  components  or  even 
charges  for  the  use  of  components.   Substantial  work  has 
been  done  in  developing  formal  techniques  for  estimating 
such  costs  [20,  2*0,  but  their  main  effect  has  been  to 
reduce  the  margin  of  error  from  outrageous  to  large.   Yet 
the  estimates  made  of  the  cost  to  implement  functions  in 
either  the  central  site  or  the  satellite  may  be  the  dominant 
factor  in  system  selection. 
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For  example,  large  systems  today  all  come  with  operating 
systems,  and  most  with  communications  handlers  as  well.   On 
the  other  hand,  few  minicomputers  come  with  the  type  of 
operating  systems  which  would  be  required  to  shift  some 
functions  from  the  central  site  to  the  satellite.   The  cost 
to  implement  such  an  operating  system  may  be  critical  to 
determining  the  cost-effectiveness  of  the  proposed  division 
of  labor. 

Of  course,  given  a  large  enough  number  of  satellite 
systems  over  which  to  share  the  total  development  costs,  the 
average  development  cost  may  be  made  arbitrarily  low.   Thus, 
an  honest  and  realistic  assessment  of  the  expected  number  of 
replications  must  also  be  made.   This  is  primarily  a  market 
research  problem. 


Comparable  Form 

The  other  problem  of  cost  analysis  is  to  express  all 
costs  in  comparable  forms.   The  assignment  of  the 
appropriate  unit  costs  to  the  use  of  the  various  resources 
is  of  critical  importance.   A  controversy  may  arise  in  the 
analysis  of  graphic  systems  over  whether  to  price  at  average 
or  incremental  costs.   For  resources  which  would  not  be 
present  if  there  were  no  graphics  system,  incremental  and 
average  costs  are  the  same.   However,  if  existing  facilities 
are  used  part  time  for  graphics,  the  incremental  costs  will 
be  less  than  the  average  costs.   If  any  tradeoff  is  to  be 
valid,  costs  for  resources  in  the  central  system  and  at  the 
satellite  must  be  comparable.   This  can  be  accomplished  by 
using,  in  all  cases",  equivalent  monthly  rental  charges  or 
average  resource  utilization  charges,  as  appropriate.* 

For  example,  a  graphics  processor  cannot  be  utilized  at 
no  cost  simply  because  it  is  owned  and  always  available.   An 
equivalent  monthly  rental  charge  must  be  determined  and  used 
to  derive  unit-time  operating  charges  so  that  they  may  be 
aggregated  with  the  charges  for  the  central  processor,  which 
are  likely  in  this  form  already.   Likewise,  file  storage 
charges  for  devices  at  either  location  must  be  represented 
in  equivalent,  non-zero  terms. 


*  Actually,  the  costs  used  for  the  analysis  may  vary 
according  to  the  purpose  of  the  analysis.   An  individual 
company  considering  adding  a  graphics  capability  to  an 
existing  facility  will  want  to  use  incremental  costs  in 
order  to  determine  if  the  addition  is  warranted.   An  analyst 
wishing  to  compare  two  alternative  graphics  systems  in  the 
abstract  would  prefer  to  use  a  measure  which  assigns  a  cost 
to  resources  irrespective  of  the  use  to  which  they  are  put. 
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Fixed  and  Variable  Costs 

The  concept  of  fixed  versus  variable  costs  is  also  a 
relevant  one.   Development  costs  such  as  design  and 
programming  costs  are  fixed,  as  are  hardware  components  in 
the  graphics  terminal  since  they  cannot  be  used  for  any 
other  purpose  (their  cost  is  fixed  regardless  of  utilization 
rate).   In  contrast,  the  use  of  central  site  resources  such 
as  storage  and  the  execution  of  programs  are  variable  costs, 
since  the  central  computer  is  shared  by  a  large  number  of 
users  who  only  pay  for  the  units  of  each  resource  as  they 
are  used.   The  costs  for  some  resources  may  be  either  fixed 
or  variable  depending  on  the  particular  arrangements.   For 
example,  communications  charges  would  be  fixed  if  a  leased 
line  were  used  and  variable  if  dial-up  facilities  were  used. 

The  significance  of  expressing  the  charges  in  this  manner 
is  that  where  fixed  charges  dominate,  total  costs  are 
relatively  constant  but  the  unit  operating  costs  for  a  Riven 
configuration  are  very  sensitive  to  the  level  of  usage.   In 
such  situations,  only  high  utilization  of  the  graphic 
equipment  can  bring  down  the  unit  cost  [26],   The 
sensitivity  of  unit  costs  to  level  of  usage  may  be  reduced 
if  fixed  costs  can  be  converted  to  variable  costs.   For 
example,  a  higher  speed  communication  link  may  make  it 
possible  to  store  programs  and  data  at  the  central  computer 
which  are  used  by  the  graphics  program  at  the  satellite. 
This  would  permit  the  amount  of  memory  at  the  satellite, 
which  represents  a  fixed  cost,  to  be  reduced  in  favor  of  use 
of  storage  at  the  central  site,  which  usually  represents  a 
variable  cost. 

However,  the  argument  goes  two  ways.   Above  a  certain 
level  of  utilization,  called  the  "breakeven  point",  it  is 
more  economical  to  convert  variable  expenses  to  fixed  ones. 
Communications  charges  are  an  excellent  example  --  above  a 
certain  level  of  utilization  it  is  cheaper  to  lease  a  full 
time  link  than  to  make  a  call  each  time.   So  estimates  of 
the  level  of  utilization  for  a  system  may  still  be  critical 
to  system  design  decisions. 

Generally,  the  problems  of  cost  analysis  for  computer 
systems  are  well-understood  (though  they  may  not  be 
well-solved).   Computer  graphics  systems  are  not  so  unique 
in  this  regard  that  they  present  any  special  problems. 
Perhaps  the  greatest  challenge  for  cost  analysis  is  to 
obtain  results  in  a  form  such  that  the  marginal  costs  of 
performance  were  obvious  or  could  be  easily  determined. 
This  would  permit  the  costs  for  improved  performance  or  the 
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savings  from  reduced  performance  to  be  more  easily  used  in 
making  design  decisions.   A  parametric  formulation 
explicitly  relating  costs  to  level  of  performance  and 
utilization  rates  could  serve  as  input  to  a  decision  model. 


5,  Cost-Effectiveness 


Cost-effectiveness  analysis  is  a  technique  used  to 
compare  alternative  systems  where  the  benefits  to  be  derived 
cannot  be  determined  quantitatively.   The  object  of 
cost-effectiveness  analysis  is  to  select  either  the  system 
with  lowest  cost  for  a  given  level  of  performance  or 
greatest  performance  for  a  given  level  of  cost.   The  method 
requires  that  either  the  costs  or  the  performance  associated 
with  all  the  systems  under  consideration  be  equal.   If  both 
costs  and  performance  are  different  for  any  two  systems  to 
be  analyzed,  then  cost-effectiveness  analysis  is  impossible 
since  the  parameters  have  different  dimensions  and  may  not 
be  compared. 

The  application  of  cost-effectiveness  analysis  is  fairly 
straightforward.   First  it  must  be  decided  which  parameter 
to  hold  constant  --  cost  or  performance.   Cost  is  frequently 
held  constant  when  there  is  no  absolute  level  of  performance 
which  is  required.   In  this  case,  a  fixed  amount  of  money  is 
made  available  and  the  analysis  seeks  the  system  with  which 
offers  the  greatest  performance,   A  measurement  criteria  is 
selected  and  the  systems  are  evaluated  according  to  that 
measure.   The  system  which  measures  the  greatest  --for 
identical  cost--  is  the  most  cost-effective.   Tradeoffs 
between  system  components  may  be  investigated  by  varying  the 
mix  of  components  subject  to  the  constraint  that  total 
system  cost  remain  the  same. 

The  alternative  approach  is  to  specify  in  advance  the 
performance  required  of  the  system.   The  system  is  selected 
which  offers  at  least  that  level  of  performance  for  the 
least  cost.   Performance  in  excess  of  the  specified 
capabilities  is  ignored  in  the  analysis.   For  such  systems, 
tradeoffs  focus  on  the  reduction  of  performance  (and  with 
it,  cost)  to  the  point  where  performance  just  equals  that 
required. 

Many  studies  have  been  published  discussing  the  tradeoffs 
involved  in  the  design  of  graphics  systems  [16,  22,  25,  31, 
32],   However,  few  published  results  show  any  serious 
attempt  to  quantify  these  tradeoffs.   Quantification  is 
necessary  if  a  true  cost-effectiveness  analysis  is  to  be 


performed.   One  of  the  first  published  results  of  a 
quantitative  analysis  of  design,  tradeoffs  was  Foley's  study 
of  satellite  graphics  systems  [11]. 


Foley  s  Analysis 

Foley  defined  optimum  design  of  a  computer  graphics 
system  as  "maximizing  a  display  system's  performance  subject 
to  a  cost  constraint."   In  his  view,  optimum  display  design 
can  be  thought  of  as  a  resource  allocation  problem.   The 
resources  are  dollars  which  can  be  allocated  to  the  purchase 
of  display  subsystems  of  differing  individual  performance. 
System  response  time  was  employed  as  the  sole  measure  of 
system  performance.   (Minimizing  response  time  means 
maximizing  performance).   Total  system  performance  is 
determined  from  a  model  of  how  the  subsystems  fit  together. 
The  parameters  of  the  model  are  functions  of  the 
capabilities  of  the  graphic  hardware  and  of  the 
computational  requirements  of  the  graphics  application.   The 
model  can  be  analyzed  using  numerical  queueing  analysis  or 
simulation  to  obtain  an  average  response  time  prediction. 
By  applying  an  optimization  procedure,  the  "best"  graphics 
system  configuration,  subject  to  a  cost  constraint,  may  be 
found  for  several  applications.   The  optimum  configurations 
are  in  turn  used  to  find  general  system  design  guidelines. 

Foley  used  a  combination  of  kernels  and  instruction  mix 
techniques  to  evaluate  the  computer  power  of  the  remote 
display  controller.   Basically  a  set  of  "display  commands" 
was  defined.   These  commands  are  at  a  level  higher  than 
machine  instructions  but  lower  than  a  high  level  language. 
The  commands  are  coded  in  the  machine  language  of  the 
controller  and  timed.   Each  display  command  timing  is 
weighted  by  a  factor  representative  of  the  relative 
frequency  of  execution  of  that  command  for  a  particular 
application.   The  greater  the  value  of  the  weighted 
instruction  execution  rate  (the  reciprocal  of  the  command 
execution  timing)  the  better  suited  is  the  corresponding 
controller  for  the  application.   The  full  account  of  Foley's 
research  [10]  indicates  that  he  also  employed  the  Adams' 
test  patterns  in  a  similar  manner  to  compare  the  suitability 
of  display  controllers  for  different  applications. 

Foley's  analyses  yielded  curves  of  the  form  shown  in 
Figure  9.   Such  curves  show  the  price  which  must  be  paid  for 
a  given  level  of  responsiveness.    Performance  is  actually 
the  reciprocal  of  response  time,  so  that  figure  9 
illustrates  the  relation  between  performance  (output)  and 
cost  (input).   The  decreasing  slope  of  the  curve  reflects 
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Figure  9.  Typical  Result  of  Foley's  Analysis 
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the  decreasing  marginal  productivity  of  additional  units  of 
cost  in  Improving  performance.   The  analyses  also  revealed 
that  different  regions  on  the  curve  were  associated  with  the 
upgrading  of  different  components  in  the  system.   Simply 
stated,  this  means  that  different  components  were  beinp- 
upgraded  when  performance  was  improving  most  rapidly  than 
when  performance  was  increasing  less  rapidly.   Thus  the 
analysis  yielded  a  set  of  decision  rules  indicating  the 
order  in  which  a  limited  amount  of  additional  money  should 
be  spent. 


Cislo's  Approach:  Simulation 

Cislo  expanded  on  Foley's  approach  by  considering  a  more 
complex  model  of  a  graphics  system  in  greater  detail  [6]. 
Foley  neglected  the  cost  of  resources  at  the  central 
computer;  Cislo's  study  includes  them.   Cislo  also  modeled  a 
multiterminal  system  in  which  each  terminal  is  used  for  a 
different  application;  this  went  beyond  Foley's  analysis. 
Finally,  Cislo  employed  simulation  in  contrast  to  Foley's 
use  of  queueing  analysis. 

Cislo's  model  of  a  graphic  system  decomposed  the 
operation  of  the  system  into  a  number  of  low  level 
operations  which  might  be  viewed  as  the  kernel  problems  of  a 
display  system.   The  activities  included  in  the  model  are 
listed  in  Table  1;  Figure  10  illustrates  the  model  itself. 
A  GPSS  program  based  on  the  model  was  written  and  used  to 
exercise  it.   The  program  simulated  operation  of  the  system 
by  generating  transactions  which  correspond  to  user 
requests.   The  interarrival  time  between  requests  was 
determined  randomly  from  a  negative  exponential 
distribution.   Similarly,  the  selection  of  the  appropriate 
routines  to  process  the  request  was  determined 
stochastically. 
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Figure  10.  Cislo's  Graphic  System  Model 
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Activity  Mnemonic 


Attention  Processing  ATNPR 

Transformation  TRANS 

Text  Processing  TXTPR 

Tracking  TRACK 

Application  Processing  ASUBR 

Data  Transmission  to  Host  DTRTH 

Host  Computer  Processing  HOST 

Data  Transmission  from  Host  DTRFH 

Data  Conversion  DCONV 

Secondary  Storage  Operation  STORG 

Update  Data  Structure  UPDTE 

Graphic  Order  Generation  GENGO 

Table  1.   Model  Activities 

The  parameters  of  the  model  are  very  similar  to  those  of 
Foley's  model  and  the  data  were  similarly  obtained. 
Published  data  were  used  for  simple  parameters  such  as 
instruction  execution  and  data  transmission  rates,  and 
simple  test  programs  were  generated  for  the  estimation  of 
such  parameters  as  the  time  required  to  generate  graphic 
orders.   Application-dependent  parameters  such  as  the 
probability  of  generating  a  given  number  of  graphic  orders 
were  much  more  difficult  to  estimate,  and  in  many  cases  were 
simply  assumed. 

In  exercising  the  model,  it  was  decided  to  hold  these 
latter  parameters  constant  while  varying  the  hardware 
parameters.   Accordingly,  the  operation  of  eight  different 
hardware  configurations,  involving  changes  to  the  main  CPU, 
display  CPU  and  data  transmission  facility,  was  simulated. 
The  results  were  in  accordance  with  Foley's  findings,  but 
the  method  accorded  much  more  detailed  descriptions  of 
system  activity.   The  expanded  version  of  the  model  which 
would  handle  multiple  independent  graphics  consoles  was 
described  but  not  implemented. 


Validity  of  Indicators 

These  techniques  which  have  just  been  described  offer 
much  hope  of  permitting  the  analysis  of  computer  graphics  to 
be  performed  on  a  much  more  systematic  and  quantitative 
basis.   The  value  of  the  analysis,  however,  is  dependent  on 
the  validity  of  the  variable(s)  which  is  optimized  as  an 
indicator  of  total  system  performance.   Accordingly,  this 
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discussion  of  cost-effectiveness  is  concluded  with  a 
discussion  of  this  question. 


Response  Time 

The  response  time  provided  to  a  user  has  long  been 
considered  a  critical  parameter  in  the  performance  of 
interactive  systems.   The  general  approach  of  system 
designers  to  this  parameter  has  either  been  to  "minimize 
it",  subject  to  available  monetary  resources,  or  to  insist 
on  a  minimum  response  time,  e.g.,  2  second  response  95$  of 
the  time,  regardless  of  the  cost.   This  approach  is  overly 
simplistic  and  may  be  far  from  cost-effective  for  a  number 
of  reasons. 

rirst  of  all,  not  all  interactive  tasks  require  the  same 
responsiveness  of  the  computer.   Miller  [231  has  shown  that 
there  are  several  classes  of  interactive  activity  with  quite 
different  response  requirements.   One  class  is  the  input  of 
data  by  various  means.   An  immediate  response  of  no  longer 
than  .1  -  .2  seconds  is  required  for  this  class  to  signify 
acceptance  of  the  data.   A  second  class  is  characterized  by 
a  user  engaged  in  high-intensity  "brainstorming"  requiring 
the  ready  access  of  data  from  his  own  "short-term  memory." 
Such  activity  requires  no  longer  than  a  two-second  response 
in  order  that  the  chain  of  thought  not  be  broken.   A  final 
class  includes  those  activities  which  complete  a  subjective 
(sub)task  or  (sub )purpose ,   More  extended  delays  (up  to  15 
seconds  or  more)  may  be  permitted  following  such  an  activity 
completion,  or  "closure",  than  in  the  process  of  obtaining  a 
closure. 

These  findings  may  have  an  important  impact  on  system 
design,  since  alternative  methods  exist  to  provide  each  type 
of  response.   For  example,  verification  feedback  and  simple 
data  manipulations  may  be  supported  on  a  so-called 
"intelligent  terminal"  with  its  own  local  mini-computer. 
Since  the  response  from  complex  application-dependent 
computations  may  be  permitted  to  be  less  rapid,  these  may  be 
provided  by  a  large  time-shared  computer,  possibly  located 
at  some  remote  location  and  connected  by  a  communications 
circuit.   A  cost-effective  design  will  assign  the  function 
of  providing  each  type  of  response  to  the  facility  which  can 
meet  the  required  response  and  meet  it  at  least  cost. 

Other  factors  may  also  be  considered  relating  to  response 
time.   A  uniform  response  time  has  come  to  be  recognized  as 
very  important  (in  contrast  to  a  response  time  which  varies 
widely  from  transaction  to  transaction)  --  so  much  so  that 
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in  some  applications  the  responsiveness  of  the  system  is 
artificially  delayed  when  it  falls  below  the  desired  value, 
so  that  the  variability  of  response  is  thereby  reduced  (at 
the  expense  of  lower  average  response  time). 

Finally,  some  experimentation  has  been  performed  into  the 
effects  of  restricting  the  free  access  to  the  computer  after 
it  has  responded  to  each  major  request  (similar  to  a 
closure)  [3]-   It  was  found  that  "users  tend  to  become 
dissatisfied  if  mild  restraint  is  placed  on  their  free 
interaction  with  the  computer."   However,  "they  also  tend  to 
problem  solve  more  effectively,  using  less  computer  time  and 
less  of  their  own  time  in  the  process."   The  authors  suggest 
that  "the  results  cast  doubts  on  the  validity  of  user 
acceptance  as  a  general  index  of  system  effectiveness." 

Clearly,  the  matter  of  the  responsiveness  required  of  a 
graphics  system  is  not  a  straightforward  question,  and  it 
does  require  serious  consideration  on  the  part  of  the  system 
designer.*   The  dilemma  is  exacerbated  by  the  demonstrated 
sensitivity  of  responsiveness  to  additional  investment  [11], 


Other  Indicators 

Another  effect  of  the  over-reliance  by  analysts  on 
response  time  as  the  sole  measure  of  performance  has  been 
the  neglect  of  a  number  of  other  important  performance 
characteristics.   These  have  been  conveniently  divided  into 
task-dependent  factors  and  human-dependent  factors  [2]. 
Task-dependent  factors  include  screen  size,  message  size, 
message  format,  erasability,  color  and  half-tone  capability 
in  addition  to  response  time.   These  factors  will  vary  with 
importance  according  to  the  particular  application;  hence  a 
weighting  approach  should  be  used  when  evaluating  systems 
according  to  these  factors. 

Human-dependent  factors,  or  more  simply  human  factors, 
permit  the  system  to  be  evaluated  on  its  usability  by  human 
beings.   Such  factors  include  brightness,  contrast, 
resolution,  readability  and  the  visual  fidelity  of  the 
display.   These  factors  are  not  as  importance  as  they  were 
in  the  early  days  of  display  system  design  when  they  were 
less-well  understood,  but  they  should  still  be  validated  for 
any  system  under  consideration.    Other  ergonomic 


*  An  interesting  application  of  response  time  data  to 
design  tradeoff  problem  is  presented  in  [33]. 
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considerations  might  also  be  important  in  special  cases,  for 
example  if  the  display  were  meant  for  proup  viewing.   Again, 
the  needs  of  the  particular  application  will  dictate 
requirements.   A  table  in  [2]  presents  a  cross  reference  of 
desirable  characteristics  for  specific  display  applications. 
Both  task-dependent  and  human-dependent  factors  are 
included. 


6.  Benefit  Analysis 


The  benefits  from  a  computer  system  arise  from  performing 
some  tasks  faster  or  qualitatively  better  than  previous 
methods.   Benefits  are  generally  measured  relative  to  the 
previous  method.   The  problem  is  to  assign  actual  dollar 
values  to  the  benefits  measured. 

Methodologies  for  performing  benefit  analysis  of  computer 
systems  are  analogous  to  the  methodologies  for  evaluating 
system  performance.   There  are  two  general  approaches: 
empirical  test  (benchmarks)  and  parametric  analysis. 

An  empirical  test  to  determine  the  benefits  of  a  given 
system  would  require  two  control  groups  and  a  typical  task. 
One  group  would  perform  the  task  according  to  the  old  or 
baseline  method;  the  other  group  would  perform  the  task  with 
the  new  system.   Two  types  of  benefits  could  be  measured 
from  such  an  experiment:  cost  savings  from  using  the  new 
system  to  achieve  the  same  level  of  performance  as  the  old 
method,  and  benefits  from  any  performance  levels  achieved  in 
excess  of  what  was  normally  accomplished  the  old  way. 

The  other  approach  of  assessing  benefits  by  parametric 
analysis  involves  decomposing  tasks  into  a  number  of 
elements.   The  benefits  from  improved  performance  in  each 
element  must  be  assessed,  and  then  the  old  and  new  system 
can  be  compared  on  each  element.   For  each  application,  the 
required  tasks  can  be  reconstructed  from  an  appropriate 
aggregation  of  elements  (where  the  parameters  specify  the 
proportions  for  each).   In  this  way  the  benefits  from  doing 
the  particular  application  with  the  new  system  can  be 
estimated. 

Unfortunately,  very  few  published  results  of  either  type 
of  study  are  extent  in  the  literature.   Productivity  claims 
of  500$  and  above  for  the  first  type  of  study  have  been 
bandied  about  at  professional  society  meetings,  but  the 
supporting  evidence  has  not  been  published,  either  because 
it  did  not  exist  in  satisfactory  form  or  because  such  data 
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was  considered  proprietary.   What  are  most  commonly  found 
are  comparisons  of  an  installed  system  with  the  old  way  of 
doing  things,  without  controls  for  any  other  variables. 
Sackman  [28]  has  presented  a  summary  of  several  such  studies 
comparing  on-line  vs.  off-line  performance  on  a  number  of 
different  tasks  [1,  9,  12,  1U,  27,  29].   The  shortcomings  of 
the  experimental  methodology  employed  in  most  of  these 
studies  has  been  critiqued  [21]. 

Occasional  studies  of  the  second  type  have  been 
published,  but  the  focus  has  generally  been  so  narrow  that 
the  results  could  not  be  generalized  [5].   An  example  of  a 
better  study  of  this  type  is  Gold's  study  of  problem-solving 
performance  in  time-sharing  versus  batch  processing  [13]* 


Gold's  Study 

Using  a  programming  language  available  in  both  batch  and 
on-line  environments,  Gold  "focused  on  a  development  of  a 
methodology  through  which  time  shared  computer  system  usage 
could  be  evaluated."   five  categories  of  variables  were 
included  in  the  methodology: 

Cost  of  using  the  system 

System  performance 

Turnaround  time 

The  mode  of  learning  resultant  from  system  use 

The  attitudes  of  the  system  users 

Principal  variables  associated  with  the  measurement  of 
the  computer  system  features  included  computer  response 
delays,  degree  of  interaction  between  the  user  and  the 
computer  system,  and  quality  of  response  in  feedback 
mechanisms  and  programming  systems. 

User  attitudes  are  defined  in  the  study  as  "the  degree  to 
which  the  computer  system  characteristics  appear  to  the  user 
to  facilitate  or  hinder  him  in  the  attainment  of  his  short 
and  long  term  objectives."   User  behavior  is  further 
characterized  in  the  study  as  "the  degree  to  which  the  user 
relegates  programmable  problem  solving  to  the  computer 
system."   Performance  for  the  purposes  of  the  study  is 
defined  as  "a  measure  of  the  output  of  the  man-computer 
system  which  is  arrived  at  independently  of  the  user's 
behavior  or  the  computer  system  used." 

The  following  are  the  significant  findings  of  the  study 
with  respect  to  user  behavior: 
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1  ,  The  users  of  the  time  sharing  system  interacted 
with  their  computer  system  more  than  three  times  as  often 
as  did  the  batch-processing  users* 

2„  There  was  no  significant  difference  between  the 
reasons  advanced  by  the  time  sharing  or  batch-processing 
users  for  initiating  computer  actions. 

3.  There  was  a  strong  relationship  between  a 
batch-processing  user's  performance  level  and  the  number 
of  computer  interactions  which  produced  usable  output. 
For  the  time  sharing  user,  the  correlation  between 
performance  and  the  number  of  sessions  with  the  time 
shared  computer  console  is  strongest. 

M.  Much  more  favorable  attitudes  toward  the  time 
shared  computer  system,  its  use  and  the  results  produced 
through  it  were  evidenced. 

With  respect  to  quality  and  cost  of  results,  the 
following  results  were  reported: 

1,  Use  of  the  time  sharing  system  resulted  in  a  higher 
level  of  objective  performance  than  use  of  the  batch 
processing  system.   Also,  the  "perception  and 
understanding  of  the  problem"  of  the  time  sharing  users 
was  evaluated  as  significantly  higher  than  that  of  the 
batch  users. 

2.  The  total  cost  of  time  sharing  and  batch  usage  did 
not  appear  to  differ  appreciably.   Higher  computer  costs 
and  lower  man  costs  were  found  with  time  sharing  usage, 
however.   There  were  indications  that  the  total  cost  for 
equivalent  objective  performance  would  have  been  less  for 
the  time  sharing  users. 


A  Suggested  Approach 

Studies  of  the  type  just  discussed  are  limited  in  that 
they  may  not  be  generalized  beyond  the  specific  sample 
activities  under  investigation.   There  has  been  no  attempt 
to  decompose  the  overall  activity  into  more  basic 
application-oriented  functions,  and  to  model  different 
activities  on  the  basis  of  these  functions. 

Application-oriented  functions  are  different  in  character 
than  the  type  of  model  activities  developed  by  Cislo.   For 
example,  text  editing  and  geometric  sketching  might  be  two 
tasks  elements  which  would  be  used  in  different  proportions 
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in  a  particular  graphics  application.   Determine  the 
benefits  of  improved  performance  on  each  of  these  tasks 
would  provide  the  necessary  data  to  determine  the  benefits 
for  an  application  comprised  solely  of  those  two  activities. 
A  sample  list  of  kernel  activities  for  benefit  analysis  is 
presented  in  Table  2.   Other  elements  need  to  be  identified 
and  analyzed. 


Text  Editing  -  original  input  and  modification 

Sketching  -  original  input  of  geometric  data 

Updating  -  modification  of  geometric  data 

Inquiry  -  transaction  processing 

Directing  -  on-line  control  of  other  operations 

Monitoring  -  of  other  operations 

Mechanization  of  Output  -  production  of  finished  drawings 

Rapid  Turnaround  -  should  be  isolated  as  a  separate 

benefit 

Table  2.   Sample  Kernel  Graphic  Tasks  for  Benefit 
Analysis 


The  paradigm  suggested  would  employ  first  analysis,  then 
experimentation,  then  synthesis  and  finally  validation  by 
means  of  experimentation  again.   The  goal. is  to  isolate  the 
individual  task  activities  in  an  application,  experimentally 
determine  the  individual  benefits  for  each,  parametrically 
aggregate  these  benefits  to  determine  the  benefits  for  the 
entire  application  and  finally  to  validate  these  results  by 
determining  directly  the  benefits  for  the  entire 
application.   Since  part  of  the  paradigm  includes 
experimentation  using  the  application  in  its  whole  or  total 
form,  an  application  of  limited  complexity  should  be  chosen. 

Having  chosen  such  an  application,  it  will  be  necessary 
to  try  to  decompose  it  into  discrete  task  areas.   This  is  a 
procedure  for  which  no  formal  rules  exist.   Common  sense  and 
some  understanding  of  what  constitute  "closures" [ 23]  in  the 
psychological  sense  will  have  to  guide  this  analysis.   The 
activities  should  be  large  enough  that  they  may  be  analyzed 
and  the  benefits  from  each  determined,  but  they  should  also 
be  sufficiently  limited  in  scope  that  they  may  be 
experimentally  manipulated  reasonably  well  and  that  they  may 
be  used  as  building  blocks  to  characterize  full 
applications . 

Having  chosen  the  task  areas,  or  kernels,  experiments 
will  be  designed  after  Gold's  model  to  determine  the 
benefits  derived  from  performing  each  task  on  a  model 
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graphics  system  as  compared  with  doing  it  some  other 
(baseline)  way.   Experimental  procedures  employing  two  or 
more  sample  groups  and  a  control  group  are  well  understood 
and  should  be  followed.   Statistical  techniques  such  as 
analysis  of  variance  may  be  useful  both  in  establishing  the 
level  of  benefits  and  validating  the  initial  choice  of  the 
groups. 

The  final  validation  of  the  approach  comes  after  an 
application  is  modeled  by  the  task  groups  and  the  benefits 
are  predicted  and  verified  by  experimentation.   This  will 
require  an  experiment  which  treats  the  entire  application  -- 
which  we  have  been  trying  to  avoid  --  but  this  is  necessary 
a  few  times  in  order  to  validate  the  approach.   If  the 
approach  can  be  validated,  experiments  on  that  scale  will 
not  be  necessary  again. 

Studies  such  as  this  are  but  a  first  step  in  the 
development  of  kernel  problems  for  the  assessment  of  benefit 
values  for  particular  systems.   Many  more  studies,  and 
preferably  a  systematic  project  with  the  stated  objective  of 
disaggregating  the  application-oriented  functions  of  a 
graphics  system  and  determining  the  benefits  of  each,  are 
required  before  a  parametric  approach  to  determining  the 
total  benefits  of  an  arbitrary  system  can  be  perfected. 


7.  Synthesis:  Cost-Benefit  Analysis 


Cost-effectiveness  analysis  is  valuable  because  it  shows 
how  to  best  allocate  limited  resources  so  as  to  optimize 
some  specific  performance  objectives.   However,  such  an 
analysis  is  inadequate  for  making  an  investment  decision. 
Such  decisions  require  that  total  system  cost  be  compared 
with  the  value  of  the  benefits  provided  by  the  system.   This 
requires  that  acceptable  levels  of  performance  be  defined, 
so  that  a  point  on  the  cost-performance  curve  (corresponding 
to  a  particular  system)  can  be  selected.   Comparing  total 
costs  with  the  benefits  to  be  derived  from  the  system 
determines  whether  or  not  the  system  selected  is 
cost-beneficial.   Unfortunately,  the  problem  is  rarely 
expressed  this  way  in  the  literature. 


Cost-Value  Technique 

One  approach  which  has  been  described  in  the  literature 
is  the  so-called  "cost-value  technique ."[ 19 ]   This  approach 
describes  a  formal  technique  for  comparing  computer  system 
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proposals  which  all  offer  slightly  different  features  beyond 
the  basic  mandatory  requirements.   The  procedure  involves 
assigning  a  value  to  each  of  these  "extras"  (expressed  in 
terms  of  dollars)  which  are  credited  to  the  cost  of  each 
proposal.   The  value  of  expansion  potential,  for  example,  is 
assessed  and  subtracted  from  the  costs  of  systems  which 
offer  it.   The  approach  is  crude  and  subject  to  all  the 
perversions  of  subjective  judgement,  but  at  least  it  forces 
the  evaluators  to  think  in  terms  of  the  benefits  to  be 
gained  from  particular  features. 


Figure  of  Merit 

In  another  vein,  Dunn  [8]  has  sought  to  define  a  figure 
of  merit  for  computer  graphics  systems  which  was  free  of 
bias  towards  any  particular  type  of  configuration  but  which 
is  applicable  to  all  types  of  configurations.   However,  Dunn 
claimed  that  "useful  measures  for  interactive  graphics 
systems  are  dependent  upon  subjective  judgements."   His 
argument  was  as  follows: 

"Any  measure  that  is  devised  must  reflect  more  than 
dollar  costs  and/or  usefulness  to  the  human  and/or 
satisfaction  achieved  via  a  particular  configuration.   An 
interactive  graphics  system  should  and  does  function  as 
an  'amplifier'  of  the  interactive  human  activities  in 
conjunction  with  the  use  of  computers.   The  amplifying 
function  affects  quality  of  results,  quantity  of  work 
accomplished,  cost-effectiveness  of  methodology, 
efficiency  of  effort,  minimization  of  elapsed  time  for 
work  efforts,  and  so  on.   Amplification  in  this  sense, 
then,  incorporates  a  variety  of  dollar  costs  along  with 
usefulness  and  satisfaction  for  the  human  user," 

He  suggested  a  figure  of  merit  called  "degree  of 
amplification"  or  "amplification  factor",  symbolized  as  "A!" 
and  based  upon  four  factors:  productivity,  degree  of 
interaction,  extent  of  graphics  capability  and  total  system 
direct  dollar  costs  per  graphics  console. 

The  productivity  factor,  P,  is  defined  as  the  change  in 
the  rate  of  units  of  acceptable  output  relative  to  the  same 
function  being  performed  via  a  non-graphics  console.   The 
units  of  acceptable  output  are  application-dependent  (e.g., 
engineering  drawings  completed), 

Graphics  capability  is  a  measurement  factor  that  reflects 
both  ease  of  use  of  the  graphics  console  and  extent  of  load 
on  supporting  computing  systems.   The  graphics  capability 
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factor,  GC  ,  is  defined  as  the  weighted  sun  of  capability 
indices  for  all  desired  features.   The  index  values  are  0  if 
a  desired  feature  was  not  available,  1  if  a  desired  feature 
was  available,  and  -1  if  a  required  or  critical  feature  was 
not  available.   The  weights  indicated  the  relative 
importance  of  each  feature. 

The  degree  of  interaction  is  measured  by  the  interaction 
quotient,  IQ,  and  is  defined  as  a  function  of  the  ratio  of 
the  effective  data  rate  of  the  connecting  channel  to  the 
interaction  data  flow  rate  requirement.   The  IQ  is  expressed 
as  a  weighted  sum  of  quotient  terms  for  each  type  of 
interaction  which  could  occur.   Each  term  has  the  form 
DR/(DO+DI),  where  DR  is  the  effective  data  rate  of  the 
channel  and  DO  and  DI  are  the  output  and  input  data  rates, 
respectively,  at  the  graphics  console  required  to  request 
and  receive  response  for  that  type  of  interactive  service 
request. 

Direct  costs  are  the  pro-rata  portions  of  system 
resources  attributable  to  graphics  support.   These  include 
the  fair  share  of  all  remote,  interconnecting  and  local 
hardware  and  software  that  is  necessary  to  implement  the 
graphics  svstem. 

Finally,  the  amplification  factor,  A!,  was  defined  as 
follows : 

dl  P  (d2  10  +  d3  GO 
A!  = 

d4  $/C 

where  P  is  the  productivity  factor;  IQ  is  the  interaction 
quotient;  GC  is  the  graphics  capability;  $  is  the  total 
system  direct  dollar  costs;  and  C  is  the  number  of  graphic 
consoles  that  may  be  concurrently  active  in  the  system.   The 
di  are  the  assigned  weights  that  reflect  the  significance 
attached  by  installations  towards  each  of  these  factors.   A! 
is  thus  a  weighted  measure  of  the  change  in  the  rate  of 
productive  output  per  dollar  of  total  graphics  system  direct 
cost  per  active  console. 


A  Cost-Benefit  Formulation 

As  has  been  indicated,  the  formulation  of  a  cost-benefit 
analysis  must  combine  the  results  of  cost-effectiveness 
studies  and  benefit  analysis.   Cost-effectiveness  analysis 
identifies  those  systems  with  the  greatest  performance  for  a 
given  level  of  investment,  or  the  least  costly  system  for  a 
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given  level  of  performance,  depending  on  how  the  analysis  is 
performed.   No  guidance  is  provided,  however,  for  the 
selection  of  a  system  from  a  group  of  systems  at  different 
levels  of  investment  and  performance,  because  there  is  no 
way  to  judge  if  incremental  improvements  in  performance  are 
worth  their  incremental  costs.   Benefit  analysis  addresses 
this  point  by  placing  a  value  on  improved  levels  of 
performance.   To  perform  cost-benefit  analysis,  the  benefits 
from  different  systems  are  systematically  compared  with 
their  respective  costs  in  a  search  for  the  system  with  the 
greatest  net  benefit  (gross  benefit  minus  cost). 

When  cost-effectiveness  analyses  are  performed  for  a 
variety  of  systems  at  different  levels  of  cost  and 
performance,  an  optimal  price-performance  function  is 
obtained.   Specifically,  this  function  is  obtained  either  by 

a)  successively  finding,  for  different  levels  of 
investment,  the  best  level  of  performance  obtainable  for 
each;  or 

b)  successively  finding,  for  different  levels  of 
performance,  the  least  costly  system  which  will  provide 
it. 

Whichever  procedure  is  followed,  the  result  is  an 
"efficient  frontier"  of  systems  which  are  optimized  at  every 
level  both  for  cost  as  a  function  of  performance  and 
performance  as  a  function  of  cost.   This  frontier  may  be 
expressed  either  as 

C  =  fKP)      (1) 

or 

P  =  f2(C)      (2) 

A  similar  frontier  may  be  obtained  from  benefit  analysis, 
representing  the  maximum  level  of  benefits  possible  for  any 
level  of  investment.   The  derivation  is  easier  to  visualize 
for  the  curve  expressing  benefit  as  a  function  of 
performance.   To  obtain  this  function,  the  benefits  for 
systems  SELECTED  FROM  THE  EFFICIENT  PRICE-PERFORMANCE 
FRONTIER  at  different  levels  of  performance  are  successively 
determined.   The  result  is  a  function  giving  the  maximum 
benefits  possible  for  each  level  of  performance.   This  may 
be  expressed  as 

B  =  f3(P)      (3) 
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Cost-benefit  analysis  seeks  the  system  with  the  maximum 
net  benefit  (the  maximum  of  benefits  minus  costs).   Thus, 
the  problem  is  to  maximize 

NB  =  B  -  C  =  f 1 (P)  -  f3(P)      (4) 

Typical  curves  for  the  functions  under  consideration  are 
illustrated  in  figure  11.   Very  little  must  be  assumed  about 
either  function  to  assure  that  a  solution  is  possible.   The 
benefit  function  must  exceed  the  cost  function  at  some  point 
or  the  system  will  never  even  be  considered  for  use.   Both 
functions  are  also  assumed  to  be  asymptotic,  for  the 
following  reasons: 

1)  The  cost  function  is  asvmptotic  in  the  sense  that 
at  some  point,  successive  increments  of  investment 
provide  successively  smaller  increments  of  Derformance. 
Economists  would  call  this  an  illustration  of  the 
decreasing  marginal  efficiency  of  capital.   In  simple 
terras,  as  performance  is  improved,  it  becomes  more  and 
more  costly  to  provide  still  better  performance. 

2)  The  benefit  function  is  asymptotic  in  the  sense 
that  at  some  point,  successive  increments  of  Derformance 
will  provide  successively  smaller  increments  of  benefit. 
Economists  refer  to  diminishing  returns;  in  simple  terms, 
the  more  of  a  good  or  service  is  provided,  the  less  is 
the  value  of  still  more  of  that  product. 

With  these  justifications  for  drawing  the  curves  as 
illustrated  in  figure  11,  an  analytic  solution  may  be 
obtained  by  the  use  of  calculus.   The  desired  maximum  occurs 
when  the  first  derivative  of  the  net  benefit  function  is 
zero : 

0  =  d(NB)/dP  =  d(f1(P))/dP  -  d(f3(P))/dP      (5) 

Subject  to  NB  >  0. 


Reformulation 

Actually,  the  cost-benefit  formulation  illustrated  thus 
far  has  been  somewhat  simplistic  in  that  it  ignores 
utilization  level.   Consideration  of  utilization  level  is  of 
primary  importance  because  of  its  effect  on  the  cost 
function  --  increased  utilization  results  in  lower  average 
costs.   Utilization  level  may  be  of  secondary  importance 
because  of  its  effect  on  benefits,  e.g.,  the  effects  of 
scale. 
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Figure  11.  Graphical  Presentation  of  Cost-Benefit  Formulation 
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In  the  reformulation  of  cost-benefit  analysis,  costs  and 
benefits  are  recognized  to  be  functions  of  both  performance 
and  utilization  level : 

C  =  f1(P,U)      (6) 

B  =  f2(P,U)      (7) 

If  the  expected  utilization  level  is  known,  it  can  be 
held  constant  for  the  analysis  and  the  previous  formulation 
is  valid.   If  it  is  desired  to  permit  utilization  level  to 
vary  in  the  analysis,  the  graphical  formulation  can  only  be 
expressed  as  a  three-dimensional  graph.   A  solution  is 
insured  by  arguments  similar  to  those  used  previously.   As 
utilization  level  increases  incrementally,  benefits  will 
also  increase,  but  at  some  point  will  increase  less  rapidly; 
likewise,  costs  will  decrease,  but  at  some  point  will 
decrease  less  rapidly.   Thus,  the  effect  of  utilization 
level  is  asymptotic  both  for  benefits  and  for  cost. 

The  analytic  solution  to  this  formulation  requires  the 
solution  of  a  simultaneous  set  of  equations  obtained  by 
setting  the  partial  derivatives  equal  to  zero: 

0  =  d(NB)/dP  =  Mf1(P,U))/dP     (8) 

0  =  d(NB)/dU  =  d(f2(P,U) )/dU      (9) 

Subject  to  NB  >  0. 

Comparison  With  Previous  Approaches 

The  cost-value  technique  may  be  viewed  as  an  attempt  to 
adjust  for  differences  in  system  performances  by  modifying 
system  cost.   The  approach  is  in  keeping  with  the  spirit  of 
cost-benefit  analysis;  it  is  just  too  simplistic  to  be 
useful  in  a  complex  case. 

Dunn's  approach  may  be  reduced  to  a  form  closer  to  the 
formulation  suggested  for  cost-benefit  analysis. 
Performance  is  actually  a  function  of  both  the  interaction 
quotient  and  graphic  capability.   Thus,  the  figure  of  merit 
reduces  to  a  performance  function  over  a  cost  function.   All 
that  is  lacking  is  to  relate  performance  to  value,  but  this 
is  facilitated  by  the  way  in  which  performance  is  expressed. 
By  expressing  performance  in  terms  of  productivity  gains,  it 
is  easy  to  determine  value  by  considering  the  value  of 
performance  achieved  the  old  way.   The  unit  benefits  of  the 
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new  system  are  just  the  unit  value  of  output  done  the  old 
way  times  the  rate  of  increase  of  productivity. 

The  real  shortcoming  off  Dunn's  approach  for  the  purposes 
of  cost-benefit  analysis  is  that  it  starts  with  the  data  it 
should  be  seeking.   Data  such  as  the  graphics  capability 
factor,  the  various  weights,  and  in  particular,  the 
productivity  factor,  are  unknown  in  advance.   The  crux  of 
the  problem  is  to  determine  the  productivity  gains  from 
using  a  particular  system.   Once  this  is  known,  and  the  cost 
for  using  the  system  is  determined,  then  the  cost-benefit 
ratio  for  the  system  can  be  determined. 


8.  Conclusions 


The  quantitative  analysis  of  computer  graphics  systems  is 
performed  at  a  very  crude  level  of  sophistication  today. 
This  is  partially  because  it  has  a  basis  in  the  evaluation 
of  computer  systems  in  general,  which  is  still  a  very 
inexact  science,  and  partially  because  it  deals  with  a 
system  involving  humans,  whose  performance  is  always 
difficult  to  assess. 

As  we  have  explained,  performance  evaluation  of  computer 
systems  in  general  and  graphics  systems  in  particular  can  be 
done  reasonably  well.   Cost  analysis  can  be  done  well  also; 
consequently  so  can  cost-effectiveness  analysis.   On  the 
other  hand,  benefit  evaluation  cannot  vet  be  done  very  well, 
so  neither  can  cost-benefit  analysis. 

Cost-effectiveness  analysis  is  necessary  but  not 
sufficient  for  making  investment  decisions.   Cost-benefit 
analysis  reauires  the  output  of  cost-effectiveness  analysis, 
since  it  requires  optimal  levels  of  performance  for  given 
levels  of  investment.   What  must  then  be  added  are  results 
from  benefit  analysis  to  determine  the  optimal  level  of 
investment.   However,  procedures  for  performing  benefit 
analysis  are  not  well-developed.   An  experimental  approach 
was  outlined  in  this  paper  which  applies  the  same  types  of 
tools  to  benefit  analysis  which  have  been  applied  to 
performance  analysis. 

Still,  cost-effectiveness  analysis  will  likely  continue 

to  be  the  principal  tool  for  choosing  among  alternative 

systems  on  the  same  order  of  cost  or  of  performance. 

Cost-benefit  analysis  is  too  gross  a  tool  to  detect  small 

differences  between  systems.  Cost-benefit  analysis  will  be 
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increasingly  used,  however,  to  place  the  "go  or  no-go" 
investment  decision  on  a  more  analytic  basis. 


Acknowledgments 


This  work  was  originally  motivated  by  Dr.  Guy  Black  of 
the  School  of  Government  and  Business  Administration,  George 
Washington  University.   The  continued  assistance- through 
many  manuscript  drafts  of  Dr.  James  Dei  Rossi  of  the 
Institute  for  Computer  Sciences  and  Technology,  National 
Bureau  of  Standards,  is  greatly  appreciated. 

Figures  5  through  9  are  copyrighted  by  the  GML 
Corporation  and  are  used  by  permission. 


BIBLIOGRAPHY 


1  Adams,  Jeanne  and  Cohen,  Leonard.   "Time-sharing  vs. 
instant  batch  processing."   <Computers  and  Automation>, 
March  1969,  pp.  33-34. 

2  Auerbach  Info,  Inc.   "Design  and  applications  of 
automated  display  systems."   <Auerbach  Standard  ED? 
Reports>,  Special  Report,  September  1968 

3  Boehm,  B.  W.,  Seven,  M.  J.  and  R.  A.  Watson. 
"Interactive  problem-solving  --  An  experimental  study  of 
'lockout'  effects."   <Proc>  Spring  Joint  Computer 
Conference,  1971,  pp.  205-210. 

4  Calingaert,  Peter.  "System  performance  evaluation: 
survey  and  appraisal,"  Communications  of  the  ACM>,  10:1 
(January  1967),  pp. 12-17. 

5  Carlisle,  James  H.   "Comparing  behavior  at  various 
computer  display  consoles  in  time-shared  legal  information." 
RAND  Corporation  Report  P-4448,  September  1970. 

6  Cislo,  Robert  A.   "Graphic  system  performance 
evaluation."   <Proc>  1972  ACM  National  Conference,  Dp. 

432-442. 


7  Cotton,  Ira  W.   "Network  graphic  attention  handling." 
<Proc>  Online  12   Symposium,  Brunei  University,  September 
1972. 

8  Dunn,  Robert  M.   "Computer  graphics:  capabilities  and 
usefulness."   <Proc>  SHARE  XL  (Denver,  Colorado,  7  March 
1973). 

9  Erikson,  W.  J.   "A  pilot  study  of  interactive  versus 
noninteractive  debugging."   System  Development  Corporation, 
TM-3296,  December  13,  1966. 

10  Foley,  James  D.   <0ptimum  design  of  computer  driven 
display  systems>.   Doctoral  Dissertation,  University  of 
Michigan,  March  1969. 

11  Foley,  James  T.   "An  approach  to  the  optimum  design 
of  computer  graphics  systems."   "(Communications  of  the  ACM> , 
14:6  (June  1971),  op.  360-390. 

12  Gold,  Michael  M.   "Methocolorv  for  evaluating 
time-shared  computer  usa^e."   Unpublished  Ph.D. 
dissertation,  1967,  Massachusetts  Institute  of  Technology, 
Alfred  P.  Sloan  School  of  Management. 

13  Gold,  Michael  M.   "Timesharing  and  batch-processinf? : 
an  experimental  comparison  of  their  values  in  a 
problem-solving  situation."   Communications  of  the  ACM>, 
12:5  (May  1969),  pp.  249-259. 

14  Grant,  E.  Eugene  and  Harold  Sackman.   "An  exploratory 
investigation  of  programmer  performance  under  on-line  and 
off-line  conditions."   <IEEE  Trans,  on  Human  Factors  in 
Electronics>,  March  1967,  pp.  33-48. 

15  Hillegass,  John  R.   "Systematic  technioues  for 
computer  evaluation  and  selection."   <Management  Services>, 
July-August  1969,  pp.  35-38. 

16  Jacobs,  Lesley  D.   "CRT  graphics  consoles  -  an  aid  to 
selection."   Rome  Air  Development  Center  Technical  Report 
71-61,  November  1971. 

17  Johnson,  R.  R.   "Needed:  a  measure  for  measure," 
<Datamation> ,  December  15,  1970,  pp.  22-30. 

18  Joslin,  Edward  0.   "Costing  the  system  design 
alternatives."   <Data  Management^  April  1971,  op.  23-27. 

19  Joslin,  Edward  0.  and  Martin  J.  Mullin.   "Cost-value 


39 


technique  for  evaluation  of  computer  system  proposals." 
<Proc>  Spring  Joint  Computer  Conference,  1964,  pp.  367-381. 

20  LaBolle,  V.   "Development  of  equations  for  estimating 
the  costs  of  computer  program  production."   System 
Development  Corporation,  June  1 Q 66  (Published  as  U.  S.  Air 
Force  Electronic  System  Division  Technical  Report  No. 
66-350), 

21  Lampson,  Butler  W.   "A  critique  of  'An  exploratory 
investigation  of  programmer  performance  under  on-line  and 
off-line  conditions.'"   <IEEE  Trans.  Human  Factors  in 
Electronics> ,  pp.  48-51. 

22  Meyer,  T.  H.  and  I.  E.  Sutherland.   "On  the  design  of 
display  processors."   <Comm.  AC.M>,  11:6  (June,  1968),  Dp. 
410-414. 

23  Miller,  Robert  B.   "Response  time  in  man-computer 
conversational  transactions,"   <Proc>  Fall  Joint  Computer 
Conference,  1968,  pp.  267-277. 

24  Nelson,  E.  A.   "Management  handbook  for  the 
estimation  of  comouter  programming  costs."   System 
Development  Corporation,  Technical  Memorandum  No.  3225, 
March  20,  1967. 

25  Poole,  Harry  H.   "Comouter  display  system  tradeoffs." 
In  <Computer  Graphics:  Techniques  and  Applications> ,  Eds.: 
Parslow,  Prowse  &  Green.   New  York:  Plenum  Press,  1969. 

26  Prince,  M,  David.   interactive  Graphics  for 
Computer-Aided  Design>.   Reading,  Massachusetts: 
Addison-Wesley ,  1971. 

27  Sackman,  Harold.   <Man-Computer  Problem  Solving>. 
Princeton,  N.  J.:  Auerbach  Publishers  Inc.,  1970. 

28  Sackman,  Harold.   <Mass  Information  Utilities  and 
Social  Excellences   Princeton,  N.  J.:  Auerbach  Publishers, 
1972. 

29  Schatzoff,  M. ,  Tsao,  R.  and  R.  Wiig.   "An 
experimental  comparison  of  time  sharing  and  batch 
processing."   <CACM> ,  May  1967,  pp.  261-65. 

30  Smith,  J.  Meredith.   "A  review  and  comparison  of 
certain  methods  of  computer  performance  evaluation." 
<Computer  Bulletin>,  12:1  (May  1868),  op.  13-18. 


40 


31  Thomas,  E.  M.  "System  considerations  for  graphic 
data  processing."  <Computers  and  Automation>,  November, 
1967,  pp.  16-19. 

32  Ward,  John  E.   "Systems  engineering  problems  in 
computer-driven  CRT  displays  for  man-machine  communication." 
<IEEE  Trans.  System  Science  and  Cybernetics> ,  3:1  (June 
1967),  pp.  47-54. 

33  Yule,  A.   "Human  response  time  in  a  graphic 
environment."   <Computer  Bull.>,  June  1972,  pp.  304-305. 


41 


NBS-114A    (REV.    7-73) 


U.S.   DEPT.  OF  COMM. 

BIBLIOGRAPHIC  DATA 
SHEET 


1.  PUBLIC  ATION  OR  REPORT  NO. 

NBS   TN-826 


2.  Gov't  Accession 
No. 


3.  Recipient's  Accession  No. 


4.  TITLE  AND  SUBTITLE 

Cost-Benefit  Analysis    of   Computer   Graphics    Systems 


5.  Publication  Date 

April    1974 


6.  Performing  Organization  Code 


7.  AUTHORS 


Ira  W.    Cotton 


8.  Performing  Organ.  Report  No. 


9.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

NATIONAL  BUREAU  OF  STANDARDS 
DEPARTMENT  OF  COMMERCE 
WASHINGTON,  D.C.  20234 


10.  Project/Task/Work  Unit  No. 

6505373 


11.  Contract/Grant  No. 

73-90042 


12.  Sponsoring  Organization  Name  and  Complete  Address  (Street,  City,  State,  ZIP) 

U.    S.    Army  Electronics   Command 

AMSEL-GG-OD 

Fort  Monmouth,    N.    J.    07703 


13.  Type  of  Report  &  Period 
Covered 

Final   7/72-6/73 


14.  Sponsoring  Agency  Code 


15.  SUPPLEMENTARY  NOTES 


16.  ABSTRACT  (A  200-word  or  less  factual  summary  ol  most  significant  information.     If  document  includes  a  significant 
bibliography  or  literature  survey,  mention  it  here.) 

This  report  assesses  the  state-of-the-art  in  cost  benefit  analyses  of 
computer  graphics  systems  and  suggests  an  approach  for  developing  improved 
methodology.   Cost-benefit  analyses  are  distinguished  from  analyses  of 
system  performance  in  that  the  latter  are  directed  at  optimizing  system 
performance  at  a  given  level  of  investment,  while  the  former  are  directed 
at  justifying  the  investment  itself. 

Computer  graphic  system  design  alternatives  are  first  outlined.   Then 
methods  of  analyzing  the  performance  and  costs  of  computer  systems  in 
general  and  graphic  systems  in  particular  are  discussed.   With  this 
information  it  is  shown  how  cost-effectiveness  analyses  may  be  performed. 
The  next  crucial  step  is  to  conduct  benefit  analysis,  an  ill-defined  art. 
The  results  of  benefit  analysis  must  be  combined  with  cost-effectiveness 
analysis  in  order  to  perform  the  desired  cost-benefit  analysis. 

An  experimental  methodology  is  suggested  for  better  performing  benefit 
analyses  of  computer  graphic  systems.   A  more  rigorous  formulation  of  the 
cost-benefit  procedure  is  then  outlined.   No  attempt  is  made  in  this  report 
to  actually  perform  such  an  analysis. 


17.  KEY  WORDS  (six  to  twelve  entries;  alphabetical  order;  capitalize  only  the  first  letter  of  the  first  key  word  unless  a  proper 
name;  separated  by  semicolons) 

Computer   graphics;    cost-benefit  analysis;    cost-effectiveness;    economics; 
performance   evaluation 


18.  AVAILABILITY  $T  Unlimited 

[ For  Official  Distribution.     Do  Not  Release  to  NTIS 

I Order  From  Sup.  of  Doc,  U.S.  Government  Printing  Office 

Washington,  D.C.  20402,  SD  Cat.  No.  C13  .46:826 

I Order  From  National  Technical  Information  Service  (NTIS) 

Springfield,  Virginia  22151 


19.  SECURITY  CLASS 
(THIS  REPORT) 


UNCLASSIFIED 


20.  SECURITY  CLASS 
(THIS  PAGE) 

UNCLASSIFIED 


21.  NO.  OF  PAGES 
47 


22.  Price 

90   cents 


USCOMM-DC    29042-P74 


NBS  TECHNICAL  PUBLICATIONS 


PERIODICALS 


JOURNAL  OF  RESEARCH  reports  National 
Bureau  of  Standards  research  and  development  in 
physics,  mathematics,  and  chemistry.  Comprehensive 
scientific  papers  give  complete  details  of  the  work, 
including  laboratory  data,  experimental  procedures, 
and  theoretical  and  mathematical  analyses.  Illustrated 
with  photographs,  drawings,  and  charts.  Includes 
listings  of  other  NBS  papers  as  issued. 

Published  in  two  sections,  available  separately: 

•  Physics  and  Chemistry  (Section  A) 

Papers  of  interest  primarily  to  scientists  working  in 
these  fields.  This  section  covers  a  broad  range  of 
physical  and  chemical  research,  with  major  emphasis 
on  standards  of  physical  measurement,  fundamental 
constants,  and  properties  of  matter.  Issued  six  times 
a  year.  Annual  subscription:  Domestic,  $17.00;  For- 
eign. $21.25. 

•  Mathematical  Sciences  (Section  B) 

Studies  and  compilations  designed  mainly  for  the 
mathematician  and  theoretical  physicist.  Topics  in 
mathematical  statistics,  theory  of  experiment  design, 
numerical  analysis,  theoretical  physics  and  chemistry, 
logical  design  and  programming  of  computers  and 
computer  systems.  Short  numerical  tables.  Issued  quar- 
terly. Annual  subscription:  Domestic,  $9.00;  Foreign, 
$11.25. 

DIMENSIONS,  NBS 

The  best  single  source  of  information  concerning  the 
Bureau's  measurement,  research,  developmental,  co- 
operative, and  publication  activities,  this  monthly 
publication  is  designed  for  the  layman  and  also  for 
the  industry-oriented  individual  whose  daily  work 
involves  intimate  contact  with  science  and  technology 
— for  engineers,  chemists,  physicists,  research  man- 
agers, product-development  managers,  and  company 
executives.  Annual  subscription:  Domestic,  $6.50;  For- 
eign, $8.25. 


NONPERIODICALS 

Applied  Mathematics  Series.  Mathematical  tables, 
manuals,  and  studies. 

Building  Science  Series.  Research  results,  test 
methods,  and  performance  criteria  of  building  ma- 
terials,  components,   systems,   and   structures. 

Handbooks.  Recommended  codes  of  engineering 
and  industrial  practice  (including  safety  codes)  de- 
veloped in  cooperation  with  interested  industries, 
professional  organizations,  and  regulatory  bodies. 

Special  Publications.  Proceedings  of  NBS  confer- 
ences, bibliographies,  annual  reports,  wall  charts, 
pamphlets,  etc. 

Monographs.  Major  contributions  to  the  technical 
literature  on  various  subjects  related  to  the  Bureau's 
scientific  and  technical  activities. 

National     Standard     Reference    Data    Series. 

NSRDS  provides  quantitative  data  on  the  physical 
and  chemical  properties  of  materials,  compiled  from 
the  world's  literature  and  critically  evaluated. 

Product  Standards.  Provide  requirements  for  sizes, 
types,  quality,  and  methods  for  testing  various  indus- 
trial products.  , These  standards  are  developed  co- 
operatively with  interested  Government  and  industry 
groups  and  provide  the  basis  for  common  understand- 
ing of  product  characteristics  for  both  buyers  and 
sellers.  Their  use  is  voluntary. 

Technical  Notes.  This  series  consists  of  communi- 
cations and  reports  (covering  both  other-agency  and 
NBS-sponsored  work)  of  limited  or  transitory  interest. 

Federal  Information  Processing  Standards 
Publications.  This  series  is  the  official  publication 
within  the  Federal  Government  for  information  on 
standards  adopted  and  promulgated  under  the  Public 
Law  89-306,  and  Bureau  of  the  Budget  Circular  A-86 
entitled.  Standardization  of  Data  Elements  and  Codes 
in  Data  Systems. 

Consumer  Information  Series.  Practical  informa- 
tion, based  on  NBS  research  and  experience,  cover- 
ing areas  of  interest  to  the  consumer.  Easily  under- 
standable language  and  illustrations  provide  useful 
background  knowledge  for  shopping  in  today's  tech- 
nological marketplace. 


BIBLIOGRAPHIC  SUBSCRIPTION  SERVICES 

The  following  current-awareness  and  literature-survey  bibliographies  are  issued  periodically  by  the 
Bureau : 

Cryogenic  Data  Center  Current  Awareness  Service  (Publications  and  Reports  of  Interest  in  Cryogenics). 
A  literature  survey  issued  weekly.  Annual  subscription:  Domestic,  $20.00;  foreign,  $25.00. 

Liquefied  Natural  Gas.   A  literature  survey  issued  quarterly.  Annual  subscription:   $20.00. 

Superconducting  Devices  and  Materials.  A  literature  survey  issued  quarterly.  Annual  subscription:  $20.00. 
Send  subscription  orders  and  remittances  for  the  preceding  bibliographic  services  to  the  U.S.  Department 
of  Commerce,  National  Technical  Information  Service,  Springfield,  Va.  22151. 

Electromagnetic  Metrology  Current  Awareness  Service  (Abstracts  of  Selected  Articles  on  Measurement 
Techniques  and  Standards  of  Electromagnetic  Quantities  from  D-C  to  Millimeter-Wave  Frequencies) .  Issued 
monthly.  Annual  subscription:  $100.00  (Special  rates  for  multi-subscriptions) .  Send  subscription  order  and 
remittance  to  the  Electromagnetic  Metrology  Information  Center,  Electromagnetics  Division,  National  Bureau 
of  Standards,  Boulder,  Colo.  80302. 

Order  NBS  publications  (except  Bibliographic  Subscription  Services) 
from:  Superintendent  of  Documents,  Government  Printing  Office,  Wash- 
ington, D.C.    20402. 


U.S.  DEPARTMENT  OF  COMMERCE 
National  Bureau  of  Standards 

Washington.  D.C.  20234 


OFFICIAL  BUSINESS 

Penalty  for  Private  Use.  $300 


POSTAGE  AND  FEES  PAID 

U.S.  DEPARTMENT  OF  COMMERCE 

COM— 215 


U.! 


O^T'O/v 


